Image acquisition using a level-indication icon

ABSTRACT

During an information-extraction technique, visual suitability indicators may be displayed to a user of the electronic device to assist the user in acquiring an image of a document that is suitable for subsequent extraction of textual information. For example, an imaging application executed by the electronic device may display, in a window associated with the imaging application, a visual suitability indicator of a tilt orientation of the electronic device relative to a plane of the document. When the tilt orientation falls within a predefined range, the electronic device may modify the visual suitability indicators to provide visual feedback to the user. Then, the electronic device may acquire the image of the document using an imaging device, which is integrated into the electronic device. Next, the electronic device may extract the textual information from the image of the document using optical character recognition.

RELATED APPLICATION

The present patent is a continuation of, and hereby claims priorityunder 35 U.S.C §120 to, pending U.S. patent application Ser. No.14/252,675, entitled “Image Acquisition Using a Level-Indication Icon,”by inventors Sammy Lee, Grace Pariante, Eugene Krivopaltsev, Sreeneel K.Maddika, Bobby G. Bray JR and Andrew B. Firstenberger, filed on 14 Apr.2014 (Attorney Docket No. INTU-148205).

BACKGROUND

The present disclosure relates to techniques for acquiring an image andextracting textual information from the acquired image. Morespecifically, the present disclosure relates to a technique foracquiring an image based on visual suitability indicators and extractingthe textual information using optical character recognition.

The widespread availability of digital cameras and cellular telephoneswith integrated imaging sensors has led to a significant increase indigital photography and associated applications that use informationfrom the acquired images. For example, after a user captures an image ofa document (such as a financial document) using a digital camera or acellular telephone, an application executing on the electronic device(such as a financial application) can extract information from the imageusing optical character recognition. Then, at least in principle, thefinancial application can use the extracted information to populatefields in a form or to perform a financial calculation.

However, in practice, the ability to extract useful information from animage (especially for use with an application) is often restricted bythe image quality. For example, depending on the image quality, errorsmay occur when extracting the information using optical characterrecognition. These errors can make it more difficult, if not impossible,for an application to subsequently use the extracted information.

Consequently, after acquiring an image, the user may need to performpost-acquisition operations, such as: editing the image to crop the areaof interest; correcting any errors that occurred in the extractedinformation; and, if the image is blurry or the area of interest was notcaptured, repeating the image-capture and/or post-acquisition operationsone or more times. Performing the image-capture and/or post-acquisitionoperations once, let alone multiple times, is time-consuming and mayconsequently limit the willingness of users to acquire images or to useapplications that leverage the information in images.

SUMMARY

The disclosed embodiments relate to an electronic device that extractstextual information from an image. During operation, the electronicdevice launches an imaging application that executes on the electronicdevice. Next, the electronic device displays, in a window associatedwith the imaging application, visual suitability indicators of a tiltorientation of the electronic device relative to a plane of a document.The user can then adjust the tilt orientation of the electronic device,and the visual suitability indicators are modified when the tiltorientation falls within a predefined range. Next, the electronic deviceacquires an image of the document using an imaging device, which isintegrated into the electronic device. Finally, the electronic deviceextracts the textual information from the image of the document usingoptical character recognition.

In some embodiments, the electronic device receives a user command toextract the information, and the imaging application is launched inresponse to the user command.

Note that the visual suitability indicators may be non-textual. Forexample, the visual suitability indicators may include: a circle in thecenter of the window, a ball whose position in the window indicates thetilt orientation, and/or corner markers proximate to the corners of thewindow. Moreover, the ball may be displayed within the circle when thetilt orientation is within the predefined range. Furthermore, the visualsuitability indicators may indicate: light intensity, an in-planeorientation of the document relative to the electronic device, and/orframing of the document in the window, and the visual suitabilityindicators are further modified when: the light intensity is sufficient,the in-plane orientation falls within a second predefined range, and/orthe document is suitably framed in the window.

Additionally, modifying the visual suitability indicators may includechanging color of a visual suitability indicator.

In some embodiments, the image is acquired without the user activatingan acquisition icon associated with the imaging device. Alternatively oradditionally, the image may be acquired after the tilt orientation fallswithin the predefined range for more than a time interval.

Moreover, after acquiring the image, the electronic device may: displaythe image to a user, receive user feedback about the image from a userand, if the user feedback rejects the image, repeat the displaying,modifying and acquiring operations to acquire another image of thedocument. Alternatively or additionally, after extracting theinformation, the electronic device receives additional user feedbackfrom the user about the accuracy of the extracted information.

Another embodiment provides a method that includes at least some of theoperations performed by the electronic device.

Another embodiment provides a computer-program product for use with theelectronic device. This computer-program product includes instructionsfor at least some of the operations performed by the electronic device.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flow chart illustrating a method for extracting informationin accordance with an embodiment of the present disclosure.

FIG. 2 is a drawing illustrating a user interface in an electronicdevice that performs the method of FIG. 1 in accordance with anembodiment of the present disclosure.

FIG. 3 is a drawing illustrating the user interface of FIG. 2 inaccordance with an embodiment of the present disclosure.

FIG. 4 is a block diagram illustrating an electronic device thatperforms the method of FIG. 1 in accordance with an embodiment of thepresent disclosure.

Note that like reference numerals refer to corresponding partsthroughout the drawings. Moreover, multiple instances of the same partare designated by a common prefix separated from an instance number by adash.

DETAILED DESCRIPTION

Embodiments of an electronic device, a technique for extractinginformation, and a computer-program product (e.g., software) for usewith the electronic device are described. During thisinformation-extraction technique, visual suitability indicators may bedisplayed to a user of the electronic device to assist the user inacquiring an image of a document that is suitable for subsequentextraction of the information. For example, an imaging applicationexecuted by the electronic device (such as a software application) maydisplay, in a window associated with the imaging application, a visualsuitability indicator of a tilt orientation of the electronic devicerelative to a plane of a document. When the tilt orientation fallswithin a predefined range, the electronic device may modify the visualsuitability indicators to provide visual feedback to the user. Then, theelectronic device may acquire the image of the document using an imagingdevice, which is integrated into the electronic device. Furthermore, theelectronic device may extract the information from the image of thedocument using optical character recognition.

By facilitating acquisition of images that are suitable for subsequentextraction of the information (e.g., using optical characterrecognition), the information-extraction technique may facilitateaccurate extraction of the information. In the process, theinformation-extraction technique may simplify the use of the imagingsensor, the application and, thus, the electronic device. Consequently,the information-extraction technique may significantly improve the userexperience when using the application and the electronic device. Inaddition, the information-extraction technique may also significantlyimprove the user experience when using other applications (such as afinancial application) that leverage the extracted information.Therefore, the information-extraction technique may increase customersatisfaction and sales of one or more applications and the electronicdevice

In the discussion that follows, a user may include: an individual (forexample, an existing customer, a new customer, a service provider, avendor, a contractor, etc.), an organization, a business and/or agovernment agency. Furthermore, a ‘business’ should be understood toinclude: for-profit corporations, non-profit corporations,organizations, groups of individuals, sole proprietorships, governmentagencies, partnerships, etc.

We now describe embodiments of the information-extraction technique,which may be performed by an electronic device (such as electronicdevice 400 in FIG. 4). FIG. 1 presents a flow chart illustrating amethod 100 for extracting information. During operation, the electronicdevice launches an imaging application (operation 112) that executes onthe electronic device. For example, a user of the electronic device maypress a physical button or activate an icon in a user interface thatlaunches the imaging application. More generally, the electronic devicemay launch the imaging application in response to an optional usercommand (operation 110) that is received from the user, such as a usercommand to extract the information.

This imaging application may instruct the user to acquire an image of adocument using an imaging device (such as a CMOS or CCD sensor), whichis integrated into the electronic device. For example, the imagingapplication may display an instruction on a display of the electronicdevice. Note that the document may include a financial document (suchas: an invoice, a paycheck, a bill, a W-2 form, a financial vehicle, orany document conveying information regarding financial information).Moreover, the image may include one or more fields in the document,which may include: an account number, an amount due, and/or a due date.

After providing the instruction and before the user acquires the imageby activating an image-activation mechanism associated with the imagingdevice (such as a physical button or a virtual acquisition icon foractivating the imaging device), or before the imaging applicationautomatically acquires the image (i.e., without user action), theimaging application displays, in a window associated with the imagingapplication (e.g., on the display), visual suitability indicators of atilt orientation (operation 114) of the electronic device relative to aplane of the document. Note that the tilt orientation may reflect theangular position of the electronic device, as determined by one or moresensors in the electronic device, e.g., an accelerometer and/or agyroscope. Therefore, the tilt orientation may be based on anorientation in a coordinate system, such as x, y, and z, and associatedangles θ, α, and γ, and/or accelerations along one or more axes.

Note that the visual suitability indicators may be non-textual. Asdescribed further below with reference to FIG. 2, the visual suitabilityindicators may include: a circle in the center of the window, a ballwhose position in the window indicates the tilt orientation, and/orcorner markers proximate to the corners of the window (which may helpthe user frame the document in the window). Furthermore, the visualsuitability indicators may indicate: light intensity, an in-planeorientation of the document relative to the electronic device, and/orframing of the document in the window.

Moreover, the imaging application modifies the visual suitabilityindicators when the tilt orientation falls within, or outside, apredefined range (operation 116). As described further below withreference to FIG. 3, the ball may be displayed within the circle whenthe tilt orientation is within the predefined range (such as 5-10°).Furthermore, the visual suitability indicators may be further modifiedwhen: the light intensity is sufficient, the in-plane orientation fallswithin a second predefined range (such as 5-10°), and/or the document issuitably framed in the window. Additionally, modifying the visualsuitability indicators may include changing color of a visualsuitability indicator.

Next, the imaging application acquires an image of the document(operation 118) using the imaging device. For example, the image may beacquired when the user performs an action (such as activating theimage-activation mechanism associated with the imaging device), orwithout the user performing an action (such as when the imagingapplication automatically acquires the image). In particular, the imagemay be acquired after the tilt orientation falls within the predefinedrange for more than a time interval (such as 0.5-3 s).

Furthermore, the imaging application extracts the information from theimage of the document using optical character recognition (operation120). However, in some embodiments the information is extracted usingintelligent character recognition.

In some embodiments, after acquiring the image (operation 118), theimaging application performs one or more additional actions (operation122). For example, the imaging application may: display the image to auser on the display, receive user feedback about the image from a userand, if the user feedback rejects the image, repeat the displaying(operation 114), modifying (operation 116) and acquiring (operation 118)to acquire another image of the document. Alternatively or additionally,after extracting the information (operation 120), the electronic devicereceives additional user feedback from the user about the accuracy ofthe extracted information.

In some embodiments of method 100, there may be additional or feweroperations. For example, the extraction of the information may beoptionally based on historical information previously extracted fromother documents. In particular, if similar information has beenpreviously extracted from another instance of the document, then thispreviously extracted information can be used during the analysis toidentify and/or to correct an error(s). Thus, if the document is apaycheck, the employer name, which was extracted from a previouspaycheck, may be used to correct an error that occurred duringextraction of the employer name from the current paycheck. Alternativelyor additionally, automatic capture of one or more images may be basedon: proper centering or framing of the document (such as using edgedetection of edges of the document within the frame), proper lighting,the focus condition and/or the tilt orientation. Moreover, the order ofthe operations may be changed, and/or two or more operations may becombined into a single operation.

By providing the visual suitability indicators (which provide anintuitive sense of when the image of the document is likely to besuitable for the subsequent extraction of the information), theinformation-extraction technique can improve the accuracy of theextracted information, and can reduce the time needed to complete theprocess. Therefore, the information-extraction technique cansignificantly improve the user experience when using the imagingapplication and the electronic device.

In an exemplary embodiment, the information-extraction techniqueleverages the capabilities of digital cameras and imaging sensors incellular telephones to acquire suitable images of documents when visiblesuitability indicators (such as the tilt orientation or angle of thedigital camera) are within one or more predefined ranges. These imagesmay be acquired without user action (i.e., before the user activates orpresses a physical button, a virtual icon or an image-activationmechanism associated with an imaging device that is normally used totake pictures).

In particular, in response to instructions from the imaging application,such as a verbal prompt or a message displayed on a display or screen,the user may point the imaging device toward a location or an area ofinterest in a document (such as a field in an invoice or a check, e.g.,an account number), and this area may be displayed in the view finder onthe screen. The imaging application executing on the electronic devicemay also display or present other visual suitability indicators (such asa graphical indication of the tilt orientation) on the screen. While theuser is pointing the imaging device toward the location on the documentand the visible suitability indicators (such as the tilt orientation)are within the one or more predefined ranges, one or more images may beacquired. For example, the imaging application and/or the electronicdevice may provide a signal to the imaging device, which then takes oneor more pictures or images of the area of interest.

In some embodiments, the resulting one or more images are presented tothe user, who provides feedback on the image quality. If one or moreimages are blurry or the desired information at one of the locations isabsent (for example, if the information isn't included in the image),one or more of the images may be re-acquired.

Subsequently, information in at least some of the images can beextracted using optical character recognition, and the extractedinformation may then be used by a financial application such as aremote-check-deposit application, financial management applications, orincome-tax application. In some embodiments, the extracted informationis presented to the user prior to its use by the imaging applicationand/or the financial application. If the user indicates the extractedinformation is incorrect, the one or more images may be reacquired.

In some embodiments, the visible suitability indicators graphicallyindicate when there is sufficient lighting (or light intensity) or not.Even if ambient light is sufficient to acquire the one or more images,the imaging application may also activate an integrated flash or lightto increase the quality of the one or more images. Thus, the flash maybe selectively triggered by the imaging application and/or theelectronic device based on a focus level that corresponds to a focus ofthe imaging device and/or the exposure setting. This may make theextraction of information in the one or more images simpler and/or moreaccurate.

Additionally, to facilitate acquisition of the one or more images, insome embodiments the displayed view finder or the image-capture zoneassociated with the imaging device (such as corner markers for the frameof the image specified by the imaging application) may be adjusted. Forexample, the view finder or the image-capture zone may be adjusted basedon a size of a field associated with a given location in the document.This field may be predefined so that the desired information associatedwith the given location is captured in the corresponding images. Forexample, if the given location is an account number, the field may be arectangular block or region that includes the account number, and theview finder or the image-capture zone may be adjusted accordingly sothat the account number is included in at least some of the images (orso that visual feedback is provided to the user so that they can framethe location properly). However, if the location is an address, the viewfinder may display a larger square to cover the three or four lines in atypical address field in a document. This adjustment of the view findermay improve the accuracy of the extracted information and, thus, maysignificantly increase user satisfaction when using the imagingapplication.

In an exemplary embodiment, a customer (John) uses the imagingapplication to extract information from a document, such as a bill. WhenJohn receives a physical bill, he may activate the imaging applicationon his cellular telephone. In response to displayed instructions, Johnmay center an account number on the bill in the view finder shown on thescreen of the cellular telephone. For example, John may position thecellular telephone so that the account number is centered within cornermarkers for a frame that is displayed on a screen. (As noted previously,the corner markers in the view finder may be repositioned based on thepredefined size of a field at the location in the bill.) Furthermore, agraphical indication of the tilt orientation of the cellular telephonemay be displayed so that John can properly level the cellular telephone.Once John has done this, the imaging application may instruct or signalthe imaging device to take one or more pictures or to acquire one ormore images of the location in the bill. As noted previously, theimaging application may also enable or disable the flash based on thefocus level and/or the exposure setting (and, more generally, based onthe lighting conditions or intensity).

After the one or more images have been acquired, the imaging applicationmay process the one or more images using optical character recognition(such as by using an optical-character-recognition engine). For example,by combining scanned text (and, more generally, extracted information)from the one or more images (which may leverage the spatial-positioninformation of the location in the bill, relative distances of points ofinterest in the one or more images from the location, and/or historicaldata, such as previously extracted information or information in afinancial history of the user), the imaging application may determinethe correct result (i.e., the account number) with a high certainty.Thus, in some embodiments historical data is used if there was a matchpreviously (i.e., if the correct information was extracted previously oris known). If yes, the previous match may be used to identify and/orcorrect an error in the extracted information.

After the information is extracted, the imaging application and/or afinancial application executing on the cellular telephone may show theresulting text to John. If he feels that this is not the data that wasneeded, John can re-point the imaging device at the bill, which mayresult in one or more additional images being acquired. Moreover, Johnmay repeat the aforementioned operations for other locations on the bill(or another document), such as locations corresponding to: the addressof the biller, the bill due date, and the amount billed. Once theimaging application has processed the one or more images, the financialapplication may have all the information needed to pay the bill.

Note that in the preceding example John may or may not have taken apicture of the whole bill. Furthermore, at least some of the processingof the one or more images may be handled on the electronic device, on aremote computer and/or via a client-server architecture. For example,image processing to determine image suitability prior to imageacquisition and/or extraction of information from an acquired imageusing optical character recognition may be performed by a server thatcommunicates with the electronic device via a network.

We now describe embodiments of a user interface that includes the visualsuitability indicators. FIG. 2 presents a drawing illustrating a userinterface 200 in an electronic device that performs method 100 (FIG. 1).This user interface includes visual indicators that provide graphicalfeedback to a user so that the user can acquire an image of documentthat can be subsequently used in optical character recognition. Ingeneral, 50% of images may not be suitable for use in optical characterrecognition because of: misalignment or skew of the image relative tothe camera or imaging device; poor lighting and/or not centering thedocument properly. Consequently, the resulting image may not be suitablefor subsequent optical character recognition. This may cause the user toskip what could have been a time-saving approach. Instead, the user maymanually enter information, which can be frustrating, is prone toerrors, and may degrade the user experience.

To address these problems, the visual indicators may specify an image‘frame’ using one or more corner markers 210. When the document isoutside of the frame, the color of the relevant corner markers 210 maybe changed. For example, in user interface 200 corner markers 210-1 and210-2 may be colored red, while corner markers 210-3 and 210-4 may begreen. This may intuitive alert the user that the document is outsidethe frame on the side specified by corner markers 210-1 and 210-2. Whenthe document is within the frame specified by corner markers 210, allcorner markers 210 may be colored green.

Similarly, visual indicator 212 may intuitively and graphically alertthe user that the lighting conditions are suboptimal (such as when thereis not enough light). Thus, visual indicator 212 may include a graphicalsymbol when there is not enough light (such as a ‘sun’ or a ‘light bulb’showing a need for more illumination or light). This graphical symbolmay be colored red. In response, the user may: turn on a light, move thedocument to a location with more light and/or activate a flash mechanismin the electronic device. In some embodiments, when there isinsufficient light, the imaging application may activate the flashmechanism without requiring user action.

As shown in FIG. 3, which presents a drawing illustrating user interface200 (FIG. 2), visual indicators 310 and 312 may be used to intuitivelyalert the user about the tilt orientation of the electronic devicerelative to the document. In particular, visual indicators 310 and 312may be a level indicator (like a carpenter's level tool). When visualindicator 310 (a ball) is outside of visual indicator 312 (a circle),the tilt orientation may be unacceptable. In this case, visualindicators 310 and 312 may be colored red. However, when visualindicator 310 (a ball) is inside of visual indicator 312 (a circle), thetilt orientation may be acceptable, and visual indicators 310 and 312may be colored green.

In some embodiments, after an image is acquired, a retake or approvalbutton may be displayed in the center of user interface 200 to requestuser feedback about the image. Additionally, user interface 200 maydisplay a percentage (such as 25%) in the center of user interface 200to indicate a percentage completion during subsequent extraction ofinformation from an image using optical character recognition.

User interface 200 may make the process of obtaining suitable conditions(such as a suitable tilt orientation) a game for the user. Moreover, inthese ways user interface 200 may assist the user in acquiringhigh-quality images that are suitable for subsequent extraction ofinformation using optical character recognition.

We now describe embodiments of the electronic device. FIG. 4 presents ablock diagram illustrating an electronic device 400. This electronicdevice includes processing subsystem 410, memory subsystem 412, andnetworking subsystem 414. Processing subsystem 410 includes one or moredevices configured to perform computational operations. For example,processing subsystem 410 can include one or more microprocessors,application-specific integrated circuits (ASICs), microcontrollers,programmable-logic devices, and/or one or more digital signal processors(DSPs).

Memory subsystem 412 includes one or more devices for storing dataand/or instructions for processing subsystem 410 and networkingsubsystem 414. For example, memory subsystem 412 can include dynamicrandom access memory (DRAM), static random access memory (SRAM), and/orother types of memory. In some embodiments, instructions for processingsubsystem 410 in memory subsystem 412 include: one or more programmodules or sets of instructions (such as one or more program modules 422or operating system 424), which may be executed by processing subsystem410. For example, the one or more program modules 422 may include: afinancial application (such as financial planning software capable ofprocessing financial information, payroll software and/or accountingsoftware), an image-acquisition (or imaging) application, and/or anextraction application. Note that the one or more computer programs mayconstitute a computer-program mechanism. Moreover, instructions in thevarious modules in memory subsystem 412 may be implemented in: ahigh-level procedural language, an object-oriented programming language,and/or in an assembly or machine language. Furthermore, the programminglanguage may be compiled or interpreted, e.g., configurable orconfigured (which may be used interchangeably in this discussion), to beexecuted by processing subsystem 410.

In addition, memory subsystem 412 can include mechanisms for controllingaccess to the memory. In some embodiments, memory subsystem 412 includesa memory hierarchy that comprises one or more caches coupled to a memoryin electronic device 400. In some of these embodiments, one or more ofthe caches is located in processing subsystem 410.

In some embodiments, memory subsystem 412 is coupled to one or morehigh-capacity mass-storage devices (not shown). For example, memorysubsystem 412 can be coupled to a magnetic or optical drive, asolid-state drive, or another type of mass-storage device. In theseembodiments, memory subsystem 412 can be used by electronic device 400as fast-access storage for often-used data, while the mass-storagedevice is used to store less frequently used data.

Networking subsystem 414 includes one or more devices configured tocouple to and communicate on a wired and/or wireless network (i.e., toperform network operations), including: control logic 416, an interfacecircuit 418 and one or more antennas 420. For example, networkingsubsystem 414 can include a Bluetooth™ networking system, a cellularnetworking system (e.g., an 3G/4G network such as UMTS, LTE, etc.), auniversal serial bus (USB) networking system, a networking system basedon the standards described in IEEE 802.11 (e.g., a Wi-Fi® networkingsystem), an Ethernet networking system, and/or another networkingsystem.

Networking subsystem 414 includes processors, controllers,radios/antennas, sockets/plugs, and/or other devices used for couplingto, communicating on, and handling data and events for each supportednetworking system. Note that mechanisms used for coupling to,communicating on, and handling data and events on the network for eachnetworking system are sometimes collectively referred to as a ‘networkinterface’ for the network system. Moreover, in some embodiments a‘network’ between the electronic devices does not yet exist. Therefore,electronic device 400 may use the mechanisms in networking subsystem 414to perform simple wireless communication between the electronic devices,e.g., transmitting advertising or beacon frames and/or scanning foradvertising frames transmitted by other electronic devices.

Within electronic device 400, processing subsystem 410, memory subsystem412, and networking subsystem 414 are coupled using bus 432. Bus 432 mayinclude an electrical, optical, and/or electro-optical connection thatthe subsystems can use to communicate commands and data among oneanother. Although only one bus 432 is shown for clarity, differentembodiments can include a different number or configuration ofelectrical, optical, and/or electro-optical connections between thesubsystems.

In some embodiments, electronic device 400 includes a display subsystem426 for displaying information on a display, which may include a displaydriver and the display, such as a liquid-crystal display, a multi-touchtouchscreen, etc.

During method 100 (FIG. 1), the financial application (and, moregenerally, an arbitrary application) may be launched and executed byprocessing subsystem 410 based on a user command. For example, the usermay launch the financial application by clicking on a physical button ina keyboard or a virtual icon associated with the financial applicationthat is displayed on display subsystem 426.

Then, the financial application provides information (such as aninstruction) to the user to point an imaging device in imaging subsystem428, which is integrated into electronic device 400, to a location on adocument. For example, the financial application may present theinformation on display subsystem 426.

As the user moves electronic device 400 in response to the displayedinformation, the image-acquisition application (which is also executedby processing subsystem 410) may display one or more visual suitabilityindicators on display subsystem 426. These visual suitability indicatorsmay be based on one or more outputs from sensor subsystem 430, e.g., anaccelerometer and/or a gyroscope integrated in electronic device 400.When the image is ready to be acquired (such as when the tiltorientation is within the predefined range), the image-acquisitionapplication may communicate a signal to the imaging device to acquireone or more images. Note that these images may be acquired withoutexplicit action by the user, such as without the user activating animage-activation mechanism associated with the imaging device. Then, theimage-acquisition application may store the one or more images in memorysubsystem 412.

In addition, the extraction application may analyze the one or moreimages to extract information proximate to the location on the document.For example, the information may be extracted using optical characterrecognition.

In some embodiments, the analysis of the one or more images is based onhistorical information previously extracted from one or more otherdocuments (such as values from previous years). For example, if similarinformation has been previously extracted from another instance of thedocument, then this previously extracted information can be used duringthe analysis.

Additionally, the extracted information may be subsequently used by thefinancial application. For example, the financial application may usethe extracted information to perform a financial calculation and/or tocomplete a financial form (such as an income-tax return).

Electronic device 400 can be (or can be included in) any electronicdevice with at least one network interface. For example, electronicdevice 400 can be (or can be included in): a personal or desktopcomputer, a laptop computer, a tablet computer, a mainframe computer, aportable electronic device (such as a cellular telephone or PDA), aserver, and/or a client computer (in a client-server architecture).Although specific components are used to describe electronic device 400,in alternative embodiments, different components and/or subsystems maybe present in electronic device 400. For example, electronic device 400may include one or more additional processing subsystems 410, memorysubsystems 412, networking subsystems 414, and/or display subsystems426. Additionally, the one or more program modules 422 may include anencryption program module. In some embodiments, one or more of thesubsystems is not be present in electronic device 400. Moreover, in someembodiments, electronic device 400 may include one or more additionalsubsystems that are not shown in FIG. 4. Also, although separatesubsystems are shown in FIG. 4, in some embodiments, some or all of agiven subsystem or component can be integrated into one or more of theother subsystems or component(s) in electronic device 400. For example,in some embodiments one or more of the one or more program modules 422may be included in operating system 424 and/or the imaging applicationmay be included in the financial application.

Moreover, the circuits and components in electronic device 400 may beimplemented using any combination of analog and/or digital circuitry,including: bipolar, PMOS and/or NMOS gates or transistors. Furthermore,signals in these embodiments may include digital signals that haveapproximately discrete values and/or analog signals that have continuousvalues. Additionally, components and circuits may be single-ended ordifferential, and power supplies may be unipolar or bipolar.

While the imaging application was used as an illustrative example of animplementation of the information-extraction technique, in general theoperations in the preceding embodiments can be implemented in a widevariety of configurations and architectures. Therefore, some or all ofthe operations in the preceding embodiments may be performed inhardware, in software or both.

In the preceding description, we refer to ‘some embodiments.’ Note that‘some embodiments’ describes a subset of all of the possibleembodiments, but does not always specify the same subset of embodiments.

The foregoing description is intended to enable any person skilled inthe art to make and use the disclosure, and is provided in the contextof a particular application and its requirements. Moreover, theforegoing descriptions of embodiments of the present disclosure havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present disclosure tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art, and the generalprinciples defined herein may be applied to other embodiments andapplications without departing from the spirit and scope of the presentdisclosure. Additionally, the discussion of the preceding embodiments isnot intended to limit the present disclosure. Thus, the presentdisclosure is not intended to be limited to the embodiments shown, butis to be accorded the widest scope consistent with the principles andfeatures disclosed herein.

What is claimed is:
 1. An electronic-device-implemented method forextracting textual information from a document, comprising: launching animaging application that executes on the electronic device; displaying,in a window associated with the imaging application, visual suitabilityindicators of a tilt orientation of the electronic device relative to aplane of the document, wherein the visual suitability indicators includea ball whose position in the window indicates the tilt orientation;modifying the visual suitability indicators when the tilt orientationfalls within a predefined range; acquiring an image of the documentusing an imaging device, which is integrated into the electronic device;and extracting the textual information from the image of the documentusing optical character recognition.
 2. The method of claim 1, whereinthe method further comprises receiving a user command to extract thetextual information; and wherein the imaging application is launched inresponse to the user command.
 3. The method of claim 1, wherein thevisual suitability indicators are non-textual.
 4. The method of claim 1,wherein the visual suitability indicators include: a circle in thecenter of the window, and corner markers proximate to the corners of thewindow.
 5. The method of claim 4, wherein the ball is displayed withinthe circle when the tilt orientation is within the predefined range. 6.The method of claim 1, wherein modifying the visual suitabilityindicators includes changing color of a visual suitability indicator. 7.The method of claim 1, wherein the image is acquired without the useractivating an acquisition icon associated with the imaging device. 8.The method of claim 1, wherein the image is acquired after the tiltorientation falls within the predefined range for more than a timeinterval.
 9. The method of claim 1, wherein, after acquiring the image,the method further comprises: displaying the image to a user; receivinguser feedback about the image from a user; and if the user feedbackrejects the image, repeating the displaying, modifying and acquiringoperations to acquire another image of the document.
 10. The method ofclaim 1, wherein, after extracting the textual information, the methodfurther comprises receiving user feedback from a user about the accuracyof the extracted textual information.
 11. The method of claim 1, whereinthe visual suitability indicators further indicate at least one of:light intensity, an in-plane orientation of the document relative to theelectronic device, and framing of the document in the window; andwherein the visual suitability indicators are further modified when atleast one of: the light intensity is sufficient, the in-planeorientation falls within a second predefined range, and the document issuitably framed in the window.
 12. A computer-program product for use inconjunction with an electronic device, the computer-program productcomprising a non-transitory computer-readable storage medium and acomputer-program mechanism embedded therein, to extract textualinformation from a document, the computer-program mechanism including:instructions for launching an imaging application that executes on theelectronic device; instructions for displaying, in a window associatedwith the imaging application, visual suitability indicators of a tiltorientation of the electronic device relative to a plane of thedocument, wherein the visual suitability indicators include a ball whoseposition in the window indicates the tilt orientation; instructions formodifying the visual suitability indicators when the tilt orientationfalls within a predefined range; instructions for acquiring an image ofthe document using an imaging device, which is integrated into theelectronic device; and instructions for extracting the textualinformation from the image of the document using optical characterrecognition.
 13. The computer-program product of claim 12, wherein thevisual suitability indicators are non-textual.
 14. The computer-programproduct of claim 12, wherein the visual suitability indicators include:a circle in the center of the window, and corner markers proximate tothe corners of the window.
 15. The computer-program product of claim 14,wherein the ball is displayed within the circle when the tiltorientation falls within the predefined range.
 16. The computer-programproduct of claim 12, wherein the visual suitability indicators furtherindicate at least one of: light intensity, an in-plane orientation ofthe document relative to the electronic device, and framing of thedocument in the window; and wherein the visual suitability indicatorsare further modified when at least one of: the light intensity issufficient, the in-plane orientation falls within a second predefinedrange, and the document is suitably framed in the window.
 17. Anelectronic device, comprising: a processor; memory; and a programmodule, wherein the program module is stored in the memory andconfigurable to be executed by the processor to extract textualinformation from a document, the program module including: instructionsfor launching an imaging application that executes on the electronicdevice; instructions for displaying, in a window associated with theimaging application, visual suitability indicators of a tilt orientationof the electronic device relative to a plane of the document, whereinthe visual suitability indicators include a ball whose position in thewindow indicates the tilt orientation; instructions for modifying thevisual suitability indicators when the tilt orientation falls within apredefined range; instructions for acquiring an image of the documentusing an imaging device, which is integrated into the electronic device;and instructions for extracting the textual information from the imageof the document using optical character recognition.
 18. The electronicdevice of claim 17, wherein the visual suitability indicators include: acircle in the center of the window, and corner markers proximate to thecorners of the window.
 19. The electronic device of claim 18, whereinthe ball is displayed within the circle when the tilt orientation fallswithin the predefined range.
 20. The electronic device of claim 17,wherein the visual suitability indicators further indicate at least oneof: light intensity, an in-plane orientation of the document relative tothe electronic device, and framing of the document in the window; andwherein the visual suitability indicators are further modified when atleast one of: the light intensity is sufficient, the in-planeorientation falls within a second predefined range, and the document issuitably framed in the window.